I am trying to use the C# .net client for Elasticsearch NEST to paginate through all the available records.
I want to get a list of all ids from the server 5000 at a time. So the very first request I get 0-5000, next request 5001-10000, then 10001-15000....
It seems that I should be using the search_after API to get the records but puzzled on how to retrieve the data.
Here is what I tried to do, but I feel that I am not understanding what I am doing and how can I make multiple requests..
var products = await elasticClient.SearchAsync<Product>(x =>
x.Source(s => s.Includes(se => se.Field(sef => sef.Id))) // all I need back is the "id" field
.Sort(srt => srt.Ascending(p => p.Id)) // we can sort the ids
.SearchAfter(5000, "get list of ids??"); // I have no idea what parameters to provide this method!
);
How can I use the .net library to loop over all available ids "5000" ids at a time?
Try this with pageNumber param:
var products = await elasticClient.SearchAsync<Product>(x =>
x.Source(s => s.Includes(se => se.Field(sef => sef.Id)))
.From(5000*(pageNumber-1))
.Size(5000)
.Sort(srt => srt.Ascending(p => p.Id))
);